Skip to content

feat(rest): let a tool see the response headers it asks for (Link pagination, rate limits) - #558

Merged
keysersoft merged 3 commits into
mainfrom
keysersoft/expose-response-headers
Sep 13, 2026
Merged

keysersoft merged 3 commits into
mainfrom
keysersoft/expose-response-headers

Conversation

@keysersoft

Copy link
Copy Markdown
Contributor

Closes #557. Came out of reviewing the Sentry adapter (#551).

What changes

  • endpointMapping.exposeHeaders: string[] (REST only, opt in per tool). Names are matched case-insensitively and returned lower-cased.
  • RestEngine.executeWithMeta() returns { body, headers }; execute() is now a thin wrapper and behaves exactly as before. Tools without exposeHeaders never touch the new path.
  • The executor attaches the headers to the tool result as _headers, and parses a Link header's rel="next" into _pagination: { nextUrl, nextCursor?, cursorParam?, prevUrl? }. No next relation → no _pagination at all, which is the "last page" signal.
  • Extras are attached after the response transform (a select can't drop them); an object body is extended, anything else is wrapped as data.
  • Audit log: still the bare body. Cache: body + headers stored together in a small envelope; entries written before this still read fine.
  • Docs: new section in docs/tool-definition.md.

Not in this PR: updating existing adapters to use it. Sentry (#551), GitHub and Shopify are the obvious candidates once this is in.

Tests: response-headers.util.spec (8), rest.engine.spec +2, dynamic-mcp-tools.spec +8 (opt-in vs not, last page, non-object body, transform ordering, audit excludes headers, cache round trip incl. legacy entries). mcp-server + engines suites: 336 green. tsc + eslint clean.

A REST tool got the body and nothing else. Right for almost every call,
wrong for list endpoints that paginate through a Link header (GitHub,
GitLab, Sentry, Shopify): the model could pass a cursor in but never
learned the next one, so every such tool was one page long. Rate-limit
headers were the other thing adapters kept wishing they could show.

A tool now opts in with endpointMapping.exposeHeaders, a list of header
names. The engine returns those (lower-cased) next to the body through a
new executeWithMeta; execute() is unchanged and every tool that did not
opt in stays on it. The executor attaches them to the result as
_headers, and reads Link for rel="next" into _pagination with nextUrl
and, when the URL carries a recognisable parameter, nextCursor and its
name. _pagination is absent on the last page: absence is the signal.

Placement: after the response transform, so a select cannot drop them;
an object body is extended, anything else is wrapped as data. The audit
log keeps the bare body. The cache stores body and headers together in a
small envelope and still reads entries written before it existed.

Closes #557
The tool test page went through executeConnectorCall, which only ever
saw the body, so a tool that asked for headers looked different there
than to a model. Same helper, same shape.
GitHub's next link carries both after= and page=. The generic list chose
after, but a tool mapping page: "$page" cannot send that back. The
parameters a tool feeds from its own inputs now come first, so
nextCursor is a value the model can pass straight back. Seen against the
live API while testing locally.
@keysersoft
keysersoft merged commit 5a4fdda into main Sep 13, 2026
8 of 12 checks passed
@keysersoft
keysersoft deleted the keysersoft/expose-response-headers branch September 13, 2026 09:23
@keysersoft keysersoft mentioned this pull request Sep 13, 2026
keysersoft added a commit to Nagharjun17/anythingmcp that referenced this pull request Sep 17, 2026
Finishing @Nagharjun17's adapter so it can go in. Their tools, descriptions and
live spec; these are the three changes from the review.

- SENTRY_ORG becomes an env var and goes into every path as {{SENTRY_ORG}}, so
  the model no longer has to know the organization slug on each call and cannot
  get it wrong. The parameter is gone from all four tools.
- The three list tools opt into exposeHeaders, which HelpCode-ai#558 made possible after
  this PR was opened. Sentry paginates with a Link header, so each response now
  carries _pagination and the cursor descriptions say to follow nextCursor until
  it disappears. Before this, every list tool was exactly one page long, which
  the instructions had to apologise for.
- The quoted adapter count moves 190 to 191 in the seven files the CI gate
  checks, which a new adapter always has to do.

The base URL stays sentry.io rather than becoming {{SENTRY_BASE_URL}} as I first
suggested: an optional placeholder that nobody fills in would be sent to the
vendor as the literal string, which is a failure mode we hit elsewhere today.
Self-hosters edit the connector's base URL after installing, and the
instructions now say so.

The live spec substitutes {{SENTRY_ORG}} itself, since it drives the engine
directly and interpolation happens a layer above.
keysersoft added a commit that referenced this pull request Sep 17, 2026
* Add Sentry adapter with four read-only tools

* feat(sentry): org as an env var, real pagination, count bumps

Finishing @Nagharjun17's adapter so it can go in. Their tools, descriptions and
live spec; these are the three changes from the review.

- SENTRY_ORG becomes an env var and goes into every path as {{SENTRY_ORG}}, so
  the model no longer has to know the organization slug on each call and cannot
  get it wrong. The parameter is gone from all four tools.
- The three list tools opt into exposeHeaders, which #558 made possible after
  this PR was opened. Sentry paginates with a Link header, so each response now
  carries _pagination and the cursor descriptions say to follow nextCursor until
  it disappears. Before this, every list tool was exactly one page long, which
  the instructions had to apologise for.
- The quoted adapter count moves 190 to 191 in the seven files the CI gate
  checks, which a new adapter always has to do.

The base URL stays sentry.io rather than becoming {{SENTRY_BASE_URL}} as I first
suggested: an optional placeholder that nobody fills in would be sent to the
vendor as the literal string, which is a failure mode we hit elsewhere today.
Self-hosters edit the connector's base URL after installing, and the
instructions now say so.

The live spec substitutes {{SENTRY_ORG}} itself, since it drives the engine
directly and interpolation happens a layer above.

---------

Co-authored-by: Matteo <keysersoft@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

REST engine: give tools access to response headers (Link pagination, rate limit info)

1 participant